Assessment of a method to detect signals for updating systematic reviews.

BackgroundSystematic reviews are a cornerstone of evidence-based medicine but are useful only if up-to-date. Methods for detecting signals of when a systematic review needs updating have face validity, but no proposed method has had an assessment of predictive validity performed.MethodsThe AHRQ Comparative Effectiveness Review program had produced 13 comparative effectiveness reviews (CERs), a subcategory of systematic reviews, by 2009, 11 of which were assessed in 2009 using a surveillance system to determine the degree to which individual conclusions were out of date and to assign a priority for updating each report. Four CERs were judged to be a high priority for updating, four CERs were judged to be medium priority for updating, and three CERs were judged to be low priority for updating. AHRQ then commissioned full update reviews for 9 of these 11 CERs. Where possible, we matched the original conclusions with their corresponding conclusions in the update reports, and compared the congruence between these pairs with our original predictions about which conclusions in each CER remained valid. We then classified the concordance of each pair as good, fair, or poor. We also made a summary determination of the priority for updating each CER based on the actual changes in conclusions in the updated report, and compared these determinations with the earlier assessments of priority.ResultsThe 9 CERs included 149 individual conclusions, 84% with matches in the update reports. Across reports, 83% of matched conclusions had good concordance, and 99% had good or fair concordance. The one instance of poor concordance was partially attributable to the publication of new evidence after the surveillance signal searches had been done. Both CERs originally judged as being low priority for updating had no substantive changes to their conclusions in the actual updated report. The agreement on overall priority for updating between prediction and actual changes to conclusions was Kappa = 0.74.ConclusionsThese results provide some support for the validity of a surveillance system for detecting signals indicating when a systematic review needs updating

Acupuncture for the Treatment of Adults with Posttraumatic Stress Disorder: A Systematic Review and Meta-Analysis

21 pagesAcupuncture has been suggested as a treatment for posttraumatic stress disorder (PTSD), yet its clinical effects are unclear. This review aims to estimate effects of acupuncture on PTSD symptoms, depressive symptoms, anxiety symptoms, and sleep quality for adults with PTSD. We searched 10 databases in January 2016 to identify eligible randomized controlled trials (RCTs). We performed random effects meta-analyses and examined quality of the body of evidence (QoE) using the GRADE approach to rate confidence in meta-analytic effect estimates. Seven RCTs with 709 participants met inclusion criteria. We identified very low QoE indicating significant differences favoring acupuncture (versus any comparator) at post-intervention on PTSD symptoms (standardized mean difference [SMD] = −0.80, 95% confidence interval [CI] [−1.59, −0.01], 6 RCTs), and low QoE at longer follow-up on PTSD (SMD = −0.46, 95% CI [−0.85, −0.06], 4 RCTs) and depressive symptoms (SMD = −0.56; 95% CI [−0.88, −0.23], 4 RCTs). No significant differences were observed between acupuncture and comparators at post-intervention for depressive symptoms (SMD = −0.58, 95% CI [−1.18, 0.01], 6 RCTs, very low QoE), anxiety symptoms (SMD = −0.82, 95% CI [−2.16, 0.53], 4 RCTs, very low QoE), and sleep quality (SMD = −0.46, 95% CI [−3.95, 3.03], 2 RCTs, low QoE). Safety data (7 RCTs) suggest little risk of serious adverse events, though some participants experienced minor/moderate pain, superficial bleeding, and hematoma at needle insertion sites. To increase confidence in findings, sufficiently powered replication trials are needed that measure all relevant clinical outcomes and dedicate study resources to minimizing participant attrition

A surveillance system to assess the need for updating systematic reviews.

BackgroundSystematic reviews (SRs) can become outdated as new evidence emerges over time. Organizations that produce SRs need a surveillance method to determine when reviews are likely to require updating. This report describes the development and initial results of a surveillance system to assess SRs produced by the Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Center (EPC) Program.MethodsTwenty-four SRs were assessed using existing methods that incorporate limited literature searches, expert opinion, and quantitative methods for the presence of signals triggering the need for updating. The system was designed to begin surveillance six months after the release of the original review, and then ceforth every six months for any review not classified as being a high priority for updating. The outcome of each round of surveillance was a classification of the SR as being low, medium or high priority for updating.ResultsTwenty-four SRs underwent surveillance at least once, and ten underwent surveillance a second time during the 18 months of the program. Two SRs were classified as high, five as medium, and 17 as low priority for updating. The time lapse between the searches conducted for the original reports and the updated searches (search time lapse - STL) ranged from 11 months to 62 months: The STL for the high priority reports were 29 months and 54 months; those for medium priority reports ranged from 19 to 62 months; and those for low priority reports ranged from 11 to 33 months. Neither the STL nor the number of new relevant articles was perfectly associated with a signal for updating. Challenges of implementing the surveillance system included determining what constituted the actual conclusions of an SR that required assessing; and sometimes poor response rates of experts.ConclusionIn this system of regular surveillance of 24 systematic reviews on a variety of clinical interventions produced by a leading organization, about 70% of reviews were determined to have a low priority for updating. Evidence suggests that the time period for surveillance is yearly rather than the six months used in this project

Assessment of a method to detect signals for updating systematic reviews.

BackgroundSystematic reviews are a cornerstone of evidence-based medicine but are useful only if up-to-date. Methods for detecting signals of when a systematic review needs updating have face validity, but no proposed method has had an assessment of predictive validity performed.MethodsThe AHRQ Comparative Effectiveness Review program had produced 13 comparative effectiveness reviews (CERs), a subcategory of systematic reviews, by 2009, 11 of which were assessed in 2009 using a surveillance system to determine the degree to which individual conclusions were out of date and to assign a priority for updating each report. Four CERs were judged to be a high priority for updating, four CERs were judged to be medium priority for updating, and three CERs were judged to be low priority for updating. AHRQ then commissioned full update reviews for 9 of these 11 CERs. Where possible, we matched the original conclusions with their corresponding conclusions in the update reports, and compared the congruence between these pairs with our original predictions about which conclusions in each CER remained valid. We then classified the concordance of each pair as good, fair, or poor. We also made a summary determination of the priority for updating each CER based on the actual changes in conclusions in the updated report, and compared these determinations with the earlier assessments of priority.ResultsThe 9 CERs included 149 individual conclusions, 84% with matches in the update reports. Across reports, 83% of matched conclusions had good concordance, and 99% had good or fair concordance. The one instance of poor concordance was partially attributable to the publication of new evidence after the surveillance signal searches had been done. Both CERs originally judged as being low priority for updating had no substantive changes to their conclusions in the actual updated report. The agreement on overall priority for updating between prediction and actual changes to conclusions was Kappa = 0.74.ConclusionsThese results provide some support for the validity of a surveillance system for detecting signals indicating when a systematic review needs updating

Enhanced recovery after surgery (ERAS) for vascular surgery: an evidence map and scoping review

Abstract Background Enhanced recovery after surgery (ERAS) interventions aim to improve patient outcomes. Vascular surgery patients have unique requirements and it is unclear which ERAS interventions are supported by an evidence base. Methods We conducted a scoping review to identify ERAS randomized controlled trials (RCTs) published in the biomedical or nursing literature. We assessed interventions for applicability to vascular surgery and differentiated interventions given at preadmission, preoperative, intraoperative, and postoperative surgery stages. We documented the research in an evidence map. Results We identified 76 relevant RCTs. Interventions were mostly administered in preoperative (23 RCTs; 30%) or intraoperative surgery stages (35 RCTs; 46%). The majority of studies reported mortality outcomes (44 RCTs; 58%), but hospital (27 RCTs; 35%) and intensive care unit (9 RCTs; 12%) length of stay outcomes were less consistently described. Conclusion The ERAS evidence base is growing but contains gaps. Research on preadmission interventions and more consistent reporting of key outcomes is needed

Programs for Care System Transitions in Mental Health: A Systematic Review

Although transitions between health care systems are common when patients move between jobs or insurers, they are especially difficult to navigate when patients with mental health conditions leave an integrated system, such as the Military Health System (MHS). The authors synthesize evidence from studies of interventions that facilitate transitions between mental health care systems, such as the transition from the MHS to the Veterans Health Administration (VHA).The authors searched multiple research databases, reference-mined bibliographies of existing reviews, and consulted with experts to identify existing evaluations of transition support interventions. Key informants helped identify pertinent populations of interest who are transitioning between health care systems.Seventeen studies evaluating different approaches met inclusion criteria. Studies reported on different outcomes, and few could be combined in aggregated analyses. Analyses showed that care transition interventions can increase outpatient mental health service use, but the overall body of evidence is limited

Elusive search for effective provider interventions: a systematic review of provider interventions to increase adherence to evidence-based treatment for depression

Abstract Background Depression is a common mental health disorder for which clinical practice guidelines have been developed. Prior systematic reviews have identified complex organizational interventions, such as collaborative care, as effective for guideline implementation; yet, many healthcare delivery organizations are interested in less resource-intensive methods to increase provider adherence to guidelines and guideline-concordant practices. The objective of this systematic review was to assess the effectiveness of healthcare provider interventions that aim to increase adherence to evidence-based treatment of depression in routine clinical practice. Methods We searched five databases through August 2017 using a comprehensive search strategy to identify English-language randomized controlled trials (RCTs) in the quality improvement, implementation science, and behavior change literature that evaluated outpatient provider interventions, in the absence of practice redesign efforts, to increase adherence to treatment guidelines or guideline-concordant practices for depression. We used meta-analysis to summarize odds ratios, standardized mean differences, and incidence rate ratios, and assessed quality of evidence (QoE) using the GRADE approach. Results Twenty-two RCTs promoting adherence to clinical practice guidelines or guideline-concordant practices met inclusion criteria. Studies evaluated diverse provider interventions, including distributing guidelines to providers, education/training such as academic detailing, and combinations of education with other components such as targeting implementation barriers. Results were heterogeneous and analyses comparing provider interventions with usual clinical practice did not indicate a statistically significant difference in guideline adherence across studies. There was some evidence that provider interventions improved individual outcomes such as medication prescribing and indirect comparisons indicated more complex provider interventions may be associated with more favorable outcomes. We did not identify types of provider interventions that were consistently associated with improvements across indicators of adherence and across studies. Effects on patients’ health in these RCTs were inconsistent across studies and outcomes. Conclusions Existing RCTs describe a range of provider interventions to increase adherence to depression guidelines. Low QoE and lack of replication of specific intervention strategies across studies limited conclusions that can be drawn from the existing research. Continued efforts are needed to identify successful strategies to maximize the impact of provider interventions on increasing adherence to evidence-based treatment for depression. Trial registration PROSPERO record CRD42017060460 on 3/29/1

Systematic review of school-based interventions to prevent smoking for girls.

BackgroundThe purpose of this review is to study the effect of school-based interventions on smoking prevention for girls.MethodsWe performed a systematic review of articles published since 1992 on school-based tobacco-control interventions in controlled trials for smoking prevention among children. We searched the databases of PubMed, Embase, Web of Science, The Cochrane Databases, CINAHL, Social Science Abstracts, and PsycInfo. Two reviewers independently assessed trials for inclusion and quality and extracted data. A pooled random-effects estimate was estimated of the overall relative risk.ResultsThirty-seven trials were included, of which 16 trials with 24,210 girls were included in the pooled analysis. The overall pooled effect was a relative risk (RR) of 0.96 (95 % confidence interval (CI) 0.86-1.08; I (2)=75 %). One study in which a school-based intervention was combined with a mass media intervention showed more promising results compared to only school-based prevention, and four studies with girl-specific interventions, that could not be included in the pooled analysis, reported statistically significant benefits for attitudes and intentions about smoking and quit rates.ConclusionsThere was no evidence that school-based smoking prevention programs have a significant effect on preventing adolescent girls from smoking. Combining school-based programs with mass media interventions, and developing girl-specific interventions, deserve additional study as potentially more effective interventions compared to school-based-only intervention programs.Systematic review registrationPROSPERO CRD42012002322

Development of the Quality Improvement Minimum Quality Criteria Set (QI-MQCS): a tool for critical appraisal of quality improvement intervention publications.

ObjectiveValid, reliable critical appraisal tools advance quality improvement (QI) intervention impacts by helping stakeholders identify higher quality studies. QI approaches are diverse and differ from clinical interventions. Widely used critical appraisal instruments do not take unique QI features into account and existing QI tools (eg, Standards for QI Reporting Excellence) are intended for publication guidance rather than critical appraisal. This study developed and psychometrically tested a critical appraisal instrument, the QI Minimum Quality Criteria Set (QI-MQCS) for assessing QI-specific features of QI publications.MethodsApproaches to developing the tool and ensuring validity included a literature review, in-person and online survey expert panel input, and application to empirical examples. We investigated psychometric properties in a set of diverse QI publications (N=54) by analysing reliability measures and item endorsement rates and explored sources of disagreement between reviewers.ResultsThe QI-MQCS includes 16 content domains to evaluate QI intervention publications: Organisational Motivation, Intervention Rationale, Intervention Description, Organisational Characteristics, Implementation, Study Design, Comparator Description, Data Sources, Timing, Adherence/Fidelity, Health Outcomes, Organisational Readiness, Penetration/Reach, Sustainability, Spread and Limitations. Median inter-rater agreement for QI-MQCS items was κ 0.57 (83% agreement). Item statistics indicated sufficient ability to differentiate between publications (median quality criteria met 67%). Internal consistency measures indicated coherence without excessive conceptual overlap (absolute mean interitem correlation=0.19). The critical appraisal instrument is accompanied by a user manual detailing What to consider, Where to look and How to rate.ConclusionsWe developed a ready-to-use, valid and reliable critical appraisal instrument applicable to healthcare QI intervention publications, but recognise scope for continuing refinement